This invention relates in general to a method for the enhancement of medical images for hardcopy prints. More particularly, it relates to such a method which compensates for the losses of image high frequency contents due to the limited capability of the output device and which mimics the image tone scale on the hardcopy prints of the trans-illuminated film on a standard diagnostic lightbox.
As picture archiving and communication systems (PACs) have been widely installed and tested in the clinical environment, a virtual radiology department with multi-facilities has been gradually established in which medical images can be accessed and viewed using softcopy displays anywhere across the entire computer network. In such a situation, hardcopy prints of medical images seem obsolete and unnecessary. However, this is not true and hardcopies will always be needed for various purposes. For example, for referral medical report an attached image can greatly facilitate the descriptions.
Photographic inkjet printers and direct thermal printers are among the best choices for non-diagnostic, referral printing due to their wide availability, low cost and portability. However, when using such a printer to directly print medical images for the referral purpose, there are several major challenges: (1) reduced image local contrast and blurred details due to the limited spatial resolution of the printer, and (2) image tone scale which is different from trans-illuminated film because of either the limited density dynamic range of the printer or the reduced viewing luminance.
The spatial frequency bandwidth of both inkjet and direct thermal printers is limited due to the halftone algorithms used in the printer technology. As a result, the printed image look quite blurred and local contrast in the image is significantly reduced. Consequently, important diagnostic details, which are often associated with the high frequency contents of the image, are undesirably suppressed. To compensate for the limited spatial resolution of the printer, it is essential to enhance the image details prior to printing.
When a medical image is printed on a reflection medium such as paper, or a trans-illuminated medium such as transparency, its tone scale is usually different from its film image. The optical density which can be produced on the medium and the image viewing luminance are two dominant factors in causing the tone scale difference. For the same image, when viewing in an indoor ambient light condition, a reflection print always looks darker in the mid-tone region than the film displayed on a lightbox, even if they are printed using the same optical density. The luminance of a lightbox used in the standard diagnostic environment is approximately 2,500 cd/m2, and the typical minimum and maximum optical densities on a film are 0.21 and 3.0, respectively. Therefore, the maximum transmitance luminance produced by the film/lightbox is more than 1500 cd/m2 and an luminance dynamic range of 620:1 can be achieved. Under such a high luminance level and wide dynamic range, a typical human observer can tell around 800 Just Noticeable Differences (JNDs) of luminance changes in an image, based on Barten""s model of the human visual system (HVS) using a standard target (H. Blume, xe2x80x9cThe act/nema proposal for a grey-scale display function standard,xe2x80x9d SPIE Medical Imaging, Vol. 2707, pp. 344-360, 1996).
On the other hand, the luminance of an indoor environment with diffusive light condition varies dramatically, from about fifty to several hundred cd/m2. The typical minimum and maximum densities on a photographic reflection print are around 0.1 and 2.2, respectively, which generate a dynamic range of 120:1 and a maximum reflectance luminance of 120 cd/m2 if a luminance of 150 cd/m2 is assumed for a surface with 100% reflectance. Again based on Barten""s model there are only about 400 JNDs that can be perceived on the reflection print. When using a photographic printer to print an image on a transparency, the produced density range is usually lower than a film. If such an image is displayed on the lightbox, a less number of JNDs can be perceived on the transparency than on the film, which make the image look flat.
Other factors associated with the photographic printers, such as quantization of the input data, further reduce the JNDs that can be perceived on a hardcopy print. Photographic printers ordinarily take only 8-bit data input and therefore can print 256 greylevels at most. On the other hand, film printers often use 12-bit data input and are able to generate sufficient greylevels for diagnostic purposes.
It is preferable to display a medical image on a hardcopy print in a consistent way and with the similar perceptual image quality of film. In the clinical field, many x-ray images are evaluated on films for primary diagnostics even if digital x-ray images exist and softcopy displays are available. Also, images from other modalities, such as Computed Tomography (CT) or Magnetic Resonance Imaging (MRI), are often printed and viewed on films. A number of reasons can be given for this including cost, legal liability, as well as the unquestionable fact that most radiologists acquired their reading skills by viewing hardcopy films. Therefore, it is highly recommended to use film images as the gold standard for visualization and generate hardcopy prints that have similar tone scale to films.
Important diagnostic details are often associated with certain image features such as edges or textures. Enhancement of these image features usually involves in increasing local contrast and sharpening edges. For this purpose, histogram equalization and unsharp masking are two major classes of techniques.
A histogram equalization technique based on the whole image content is suitable to enhance the global image contrast instead of local contrast, and it sometimes attenuates the contrast of the image regions which have scarce population. On the other hand, a histogram equalization technique based on the local image content is designed to enhance the local contrast. However, it is computationally expensive (R. C. Gonzalez and R. E. Woods, Digital Image Processing, pp. 173-189, Addison-Wesley Publishing Company, 1993). In addition, histogram equalization techniques do not sharpen edges in order to address the problem caused by the limited spatial resolution of a printer.
An unsharp masking technique, broadly speaking, consists of decomposing an image into a low frequency component and a high frequency component, manipulating either component individually then combining them together. Since edges and texture in an image are usually the high frequency content, edge sharpening and local contrast enhancement can be achieved by boosting the high frequency component.
Various approaches have been developed for unsharp masking technique to decide and control the low and high frequency components of an image. The high frequency component can be determined either using a linear or a nonlinear high pass filter. Certain quadratic operators are commonly used as nonlinear filters to conduct the high pass operation (G. Ramponi, N. Strobel, S. K. Mitra, and T-H. Yu, xe2x80x9cNonlinear unsharp masking methods for image contrast enhancement,xe2x80x9d Journal of Electronic Imaging, Vol. 5, No. 3, pp. 353-366, 1996). On the other hand, since the high frequency component of an image can be computed as the difference between the original image and a low pass filtered version of that image, a median filter, which is nonlinear, can be used for low pass filtering to obtain the high frequency component (R. C. Gonzalez and R. E. Woods, Digital Image Processing, pp. 191-195, Addison-Wesley Publishing Company, 1993). However, the computation is usually expensive for a nonlinear filter with a large kernel. A convolution operator with positive coefficients around its center and negative coefficients in the outer periphery, for example an Laplacian filter, constitutes a linear high pass filter. Due to the fact that a linear filter is easy to implement and fast in execution, it has been widely used in unsharp masking techniques. In practice, a rectangle operator with all the coefficients being equal to unity is the most commonly used linear low pass filter, and the high frequency component of an image is obtained with the subtraction from the original image of the low frequency component.
For the unsharp masking techniques, many methods have been developed to combine the high frequency and low frequency components of an image. Most of them leave the low frequency component unchanged and manipulate only the high frequency component. For example, the high frequency component is multiplied by a boost factor a, then is added to the original image, as given by Eq. 1,
P=O+xcex1xc2x7H,xe2x80x83xe2x80x83(1)
where P is the processed image, O is the original image and H is the high frequency component, respectively. Murakami disclosed a method to determine the boost factor a based on the observing distance and the spatial frequency content of the image (U.S. Pat. No. 5,204,919, issued 1990, inventor Murakami). The boost factor can also be decided based on the statistics of the pixel values of the images, as proposed by Mohamoodi (U.S. Pat. No. 4,571,635, issued 1984, A. B. Mohmoodi). To improve the computation efficiency for hardware execution, Joyce used an annular mean and an annular standard deviation to compute the boost factor (U.S. Pat. No. 4,941,190, issued 1988, inventor T. H. Joyce). Morishita et al proposed to divide the entire image into a plurality of regional images, and for each regional image the boost factor is determined based on either the pixel value of the original image or the standard deviation of the regional pixel values (U.S. Pat. No. 4,794,531, issued 1987, inventors K. Morishita, et al). Song has patented two algorithms, in which the boost factor is determined adaptively based on the variance of the surrounding pixel values, the noise power spectrum of the image system, and the patterns in the image (U.S. Pat. No. 4,783,840, issued 1987, inventor W. J. Song; and U.S. Pat. No. 5,038,388, issued 1989, inventor W. J. Song). Hishinuma et al computed the boost factor based on the median value of a group of pixels within the unsharp mask of a predetermined size (U.S. Pat. No. 4,747,052, issued 1985, inventors K. Hishinuma et al). Kato et al used a boost factor that varies with either the original or its low frequency component (U.S. Pat. No. 4,315,318, issued 1979, inventors H. Kato et al; and U.S. Pat. No. 4,317,179, issued 1979, inventors H. Kato et al). There are a handful of other algorithms that further combine the boost factor with the high frequency component together, which is broadly described by Eq. 2,
P=O+f(H),xe2x80x83xe2x80x83(2)
where f(H) is a function of H. Haraki et al patented an algorithm in which f(H) varies in a certain way with the absolute value of H (U.S. Pat. No. 5,369,572, issued 1992, inventors Haraki et al). Ishida et. al recommended f(H) increase monotonically with H (U.S. Pat. No. 4,387,428, issued 1980, inventors Ishida et al; and U.S. Pat. No. 4,346,409, issued 1980, inventors Ishida et al). All the above mentioned methods (Eqs. 1-2) are effective in sharpening edges and enhancing local contrast, but none of them tried to address the problems caused by the reduced viewing luminance and the limited dynamic range of the output device, i.e., to propose a solution to compress the dynamic range of the image and adjust the image tone scale.
To compress the dynamic range of an image, the low frequency component can also be processed, which is mathematically described by Eq. 3,
P=Oxe2x88x92xcex2xc2x7L,xe2x80x83xe2x80x83(3)
where xcex2 is an attenuation factor. Nakazawa et al used a positive xcex2 which is less than unity to reduce the dynamic range of the low frequency component, while leaving the high frequency unchanged (U.S. Pat. No. 5,319,719, issued 1992, inventors Nakazawa et al). Further improvement of this method includes combining xcex2 and L together, as given by Eq. 4,
P=O+g(L),xe2x80x83xe2x80x83(4)
where g(L) is a function of L. Nakazawa et al tried a method in which g(L) is a function that decreases as L increases (U.S. Pat. No. 5,454,044, issued 1994, inventor Nakajima). More specifically, Nakajima recommended g(L) should decrease monotonically as the value of L increases (U.S. Pat. No. 5,454,044, issued 1994, inventor Nakajima; and U.S. Pat. No. 5,608,813, issued 1995, inventor Nakajima). Another algorithm by Nakazawa et al uses a very broad function description (Eq. 5),
P=A(O)+B(L),xe2x80x83xe2x80x83(5)
where A(O) is a function of O only, and B(L) is a function of L only (U.S. Pat. No. 5,604,780, issued 1995, inventors Nakazawa et al). All the methods in Eqs. 3-5 can be used to effectively compress the image dynamic range, however, they do not sharpen edges and enhance local contrast in order to compensate for the limited spatial resolution of the output devices.
Several early publications also revealed algorithms that process both the low frequency and high frequency components. Tsuchino et al used an algorithm based on Eq. 6 (U.S. Pat. No. 5,493,622, issued 1994, inventors Tsuchino et al),
P=O+f1(L1)+g(Oxe2x88x92L2).xe2x80x83xe2x80x83(6)
This is a good approach in that it uses f1(L1) to compress the dynamic range in a low frequency region L1 of the original image, at the same time uses g(Oxe2x88x92L2) to boost a different high frequency region H2=Oxe2x88x92L2. Muka et al tested their algorithm in which the attenuation of the low frequency and the amplification of the high frequency are based on the input/output luminance dynamic ranges, the spatial frequency bandwidth of the output device and knowledge of the human visual system (U.S. Pat. No. 5,774,599, issued Jun. 30, 1998, inventors Muka et al). However, no adaptive dependency of attenuation/amplification on individual image pixel value is used by this algorithm. Nevertheless, the image tone scale is still one of the problems to be addressed.
Other prior art on image enhancement and tone scale adjustment includes the one from Varies (F.P.P.D. Varies, xe2x80x9cAutomatic, adaptive, brightness independent contrast enhancement,xe2x80x9d Signal Processing, No. 21, pp. 169-182, 1990). In his work the author used logarithmic and exponential calculations for dynamic range transform and applied adaptive local difference operator in the logarithmic domain for local contrast enhancement. While good image quality is expected, the computation is complicated and may be formidable for a large image of 2500xc3x972048 pixels.
In view of the above mentioned drawbacks of the related prior art, the objective of the present invention is to provide a computationally efficient method of digital image processing, especially edge sharpening and tone scale adjustment of medical images, for hardcopy prints which (1) are produced by output devices with limited capabilities, such as a limited spatial resolution and dynamic range, and/or (2) are viewed in a limited viewing conditions, such an indoor ambient light environment.
According to a feature of the present invention, there is provided a method for edge sharpening and tone scale adjustment of digital images compromising the steps of: (1) obtaining a medical image in digital format; (2) conducting a plurality of operations for Human Visual System (HVS)-based edge enhancement with each operation having a uniquely specified frequency amplification function and range, such that edges are more enhanced in the shadow region than in the highlight region; (3) adjusting the image tone scale to match that of a trans-illuminated film: for a reflection print, the local contrast in the shadow region is enhanced while that in the highlight region is suppressed, and for a transparency print, the local contrast in the shadow region is suppressed while that in the highlight region is enhanced; (4) sending the processed image to the output device for hardcopy prints.
The present invention has the following advantages:
1. Edges sharpening of medical images is human visual system based; edges are more enhanced in the shadow region than in the highlight region, therefore a uniformly perceived edge enhancement can be achieved across the entire image while the noise can be suppressed to a desirable level.
2. Tone scale adjustment of medical images is human visual system based. An image tone scale similar to that of a film can be achieved while the dynamic range of the input image is compressed simultaneously.
3. Edge enhancement and image tone scale can be adjusted based on the observer""s preference, diagnostic regions of interest, and the characteristics of the output devices.